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[i] During solar cycle 22, a very intense geomagnetic storm on 13 March 1989 contributed to the 
collapse of the Hydro-Quebec power system in Canada. This event clearly demonstrated that 
geomagnetic storms have the potential to lead to blackouts. This paper addresses whether geomagnetic 
activity challenged power system reliability during solar cycle 23. Operations by PJM Interconnection, 
LLC (hereafter PJM), a regional transmission organization in North America, are examined over the 
period 1 April 2002 through 30 April 2004. During this time PJM coordinated the movement of wholesale 
electricity in all or parts of Delaware, Maryland, New Jersey, Ohio, Pennsylvania, Virginia, West 
Virginia, and the District of Columbia in the United States. We examine the relationship between a 
proxy of geomagnetically induced currents (GICs) and a metric of challenged reliability. In this study, 
GICs are proxied using magnetometer data from a geomagnetic observatory located just outside the PJM 
control area. The metric of challenged reliability is the incidence of out-of-economic-merit order 
dispatching due to adverse reactive power conditions. The statistical methods employed make it 
possible to disentangle the effects of GICs on power system operations from purely terrestrial factors. 
The results of the analysis indicate that geomagnetic activity can significantly increase the likelihood 
that the system operator will dispatch generating units based on system stability considerations rather 
than economic merit. 
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1. Introduction 

[ 2 ] During solar cycle 22, a very intense geomagnetic 
storm on 13 March 1989 led to the collapse of the Hydro- 
Quebec power system in Canada. This event clearly dem- 
onstrated that geomagnetic storms have the potential to 
lead to blackouts. This vulnerability exists because geo- 
magnetically induced currents (GICs) can sharply degrade 
the performance of critical transmission infrastructure. 
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[ 3 ] No major blackouts were attributed to geomagnetic 
activity during solar cycle 23. However, this may under- 
state the reliability challenge if system operators were able 
to avoid blackouts by undertaking actions that had non- 
trivial costs. The number of blackouts over a solar cycle 
may also be an unreliable metric of the potential reliability 
challenge if the vulnerability of the power grid is contin- 
gent on system conditions and the number of large geo- 
magnetic storms is small. 

[ 4 ] Indicative of space weather's perceived threat to the 
reliability of the electric power system, the North American 
Electric Reliability Corporation (NERC) has advised the 
electric power industry to monitor the power system for 
geomagnetic effects when the National Oceanic and Atmo- 
spheric Administration (NOAA) Space Weather Prediction 
Center (SWPC) issues a K-index warning of Kp of 6 or 
higher [Rollison et al., 2011]. The SWPC website indicates 
that this is approximately equivalent to a maximum change 
in the horizontal component of the geomagnetic field that 
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equals or exceeds 120 nT/min (http://www.swpc.noaa.gov/ 
info/Kindex.html). At the Kp = 6 level of geomagnetic 
activity, system operators are advised to increase reserves of 
reactive power, decrease loading on susceptible equipment, 
and consider increasing import capacity (see "Background 
for alert: Preparing for geo-magnetic disturbances," 2011, 
http://www.nerc.com/docs/pc/gmdtf/GMD_Background_ 
Draft_05062011_CLEAN.pdf). More extensive measures 
are advised when more severe geomagnetic conditions 
are expected. 

[ 5 ] This paper addresses whether geomagnetic activity 
was actually a challenge to power system reliability during 
solar cycle 23. Do the data support NERC's advisory or 
does the link between space weather and the power grid 
become an issue only when the magnitude of the geo- 
magnetic storm reaches or exceeds that of March 1989? 

M The paper uses data from the PJM regional trans- 
mission organization (RTO) in the mid-Atlantic region of 
the United States to examine the relationship between 
geomagnetically induced currents (GICs) and challenged 
power system reliability. This research follows Forbes 
and St. Cyr [2008], who presented evidence that various 
metrics of power grid operations in 12 geographically 
disparate electricity grids were statistically related with 
a proxy for GICs. More recently, Forbes and St. Cyr 
[2010] have presented multivariate statistical evidence 
that GICs contribute to constraints in PJM's 500 kilovolt 
(kV) transmission system. 

[ 7 ] The remainder of the paper is organized as follows. 
Section 2 discusses reactive power, a form of power that is 
critical to the stability of the transmission system. It also 
discusses "reactive off-cost" operations, which are a set of 
procedures PJM implements when there are inadequate 
levels of reactive power. The incidence of these events is 
the metric of challenged reliability considered in this paper. 
Section 3 discusses the data employed in the study. Section 4 
presents a chi-square analysis of the GIC proxy and the 
incidence of reactive off-cost events in the PJM transmis- 
sion system. Section 5 presents a multivariate model of 
reactive off-cost events that takes into account possible 
confounding factors. Section 6 reports the estimation 
results and also offers evidence that the reactive power 
vulnerability of the transmission system is not constant 
over time but instead varies with power system conditions. 
Section 7 summarizes the findings and discusses research 
that could mitigate the overall reliability challenge by 
forecasting the vulnerability of the transmission system. 

2. Reactive Power and the Reliability 
of the Power Grid 

[8] PJM Interconnection coordinated the dispatch of 
76,000 megawatts (MW) of generating capacity over 
approximately 32,000 km of transmission lines in all or parts 
of Delaware, Maryland, New Jersey, Ohio, Pennsylvania, 
Virginia, West Virginia, and the District of Columbia as 
of 30 April 2004. The sample period for this analysis is 


1 April 2002 through 30 April 2004. During this period, 
PJM operated both real-time and day-ahead markets for 
energy. Prices in both markets are location-based, which 
means the prices will be equal across locations when the 
transmission system is uncongested but can vary substan- 
tially from one location to another when there are trans- 
mission constraints. For example, transmission constraints 
can lead to significant differences in the real-time price at 
PJM's Eastern Hub compared to its Western Hub. 

[ 9 ] A substantial portion of PJM's generating capacity is 
accounted for by coal-fired power plants in the western 
part of the control area while demand centers are in the 
east. Exclusive of the environmental damage associated 
with coal-fired generation, the generating costs of pro- 
ducing electricity from coal are relatively low. Thus, market 
forces favor the dispatch of coal-fired electricity and the 
associated large transfers of electricity from the western 
portion of the control area to the eastern demand centers. 
For example, over the sample period, the average transfers 
of electricity at PJM's western interface were approxi- 
mately 4800 megawatt hours (MWh) while the transfer 
limits, the values beyond which reactive and voltage cri- 
teria are violated, averaged approximately 5800 MWh. 
The averages of the transfers and the transfer limits at 
PJM's eastern interface were approximately 5000 and 
6000 MWh, respectively. To put these values in perspec- 
tive, based on data downloaded from the European Net- 
work of Transmission System Operators for Electricity 
(http://www.entsoe.net/home.aspx), the 2010 net transfer 
capacity of the transmission lines between France and 
Germany was approximately 2800 MWh for exports from 
France to Germany and 2850 for exports from Germany to 
France. 

[ 10 ] High-voltage electricity transmission systems almost 
exclusively employ alternating current technology, with 
notable exceptions such as the interconnector between 
England and France, the cable that links the North and 
South Islands of New Zealand, and the Cross Sound cable 
project that links Long Island with Connecticut. For 
example, as of 31 December 2004, approximately 254,397 
of the 258,634 km of high-voltage transmission lines 
(230 kV and above) in the United States were accounted for 
by alternating current technology [North American Electric 
Reliability Corporation, 2005]. In an alternating current sys- 
tem, voltage and current oscillate up and down 50 times per 
second in most of the world (60 times per second in North 
America). As a result, the power transmitted on single 
transmission line also pulsates up and down around 
the average value [Sauer, 2003]. This average value is a 
measure of "real" power. Another form of power known 
as reactive power is also supplied or consumed depend- 
ing on whether current peaks before or after voltage 
[U.S. Federal Energy Regulatory Commission (FERC), 2005]. 
Reactive power maintains the voltages required for system 
stability and thus is critical to the delivery of real power to 
consumers. Sources of reactive power include generators 
and capacitors. Reactive power is consumed by 
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transmission lines, transformers, and motors. Excessive 
reactive power consumption has the potential to lead to 
voltage collapse and system instability. A key attribute 
of reactive power is that its consumption can increase 
significantly with the distance transported [FERC, 2005]. 
This attribute is critical to the modeling of reactive power 
deficiencies. 

[ 11 ] According to FERC [2005], voltage collapse due to 
inadequate reactive power has contributed to a number of 
blackouts. Among those cited by FERC were blackouts on 
2 July 1996 and 10 August 1996 on the West Coast of the 
United States; 19 December 1978 in France; 23 July 
1987 in Tokyo, Japan; 28 August 2003 in London, 
England; 23 September 2003 in Sweden and Denmark; 
and 28 September 2003 in Italy. FERC even notes that 
voltage collapse due to inadequate reactive power was a 
contributing factor to the blackout experienced by Hydro- 
Quebec on 13 March 1989. 

[ 12 ] The U.S. -Canada Power System Outage Task Force 
investigated the causes of the 14 August 2003 blackout in 
the Northeast United States and Canada. The task force 
concluded that inadequate management of reactive power 
was a contributing factor in the blackout [U.S. -Canada 
Power System Outage Task Force, 2004, p. 18]: 

The Ohio phase of the August 14, 2003, blackout was caused 
by deficiencies in specific practices, equipment, and human 
decisions by various organizations that affected conditions and 
outcomes that afternoon — for example, insufficient reactive 
power was an issue in the blackout, but it was not a cause in itself. 
Rather, deficiencies in corporate policies, lack of adherence to 
industry policies, and inadequate management of reactive power 
and voltage caused the blackout, rather than the lack of reactive 
power. 

[ 13 ] A 23 September 2003 article in the New York Times 
by Richard Perez-Pena and Eric Lipton is somewhat more 
illuminating on the role of reactive power in the 14 August 
2003 blackout [Perez-Pena and Lipton, 2003, paragraph 5]: 

Experts now think that on Aug. 14, northern Ohio had a severe 
shortage of reactive power, which ultimately caused the power 
plant and transmission line failures that set the blackout in 
motion. Demand for reactive power was unusually high because 
of a large volume of long-distance transmissions streaming 
through Ohio to areas, including Canada, than needed to import 
power to meet local demand. But the supply of reactive power 
was low because some plants were out of service and, possibly, 
because other plants were not producing enough of it. 

[ 14 ] PJM experienced significant reactive power/low 
voltage challenges in 1999. For example, very low voltages 
were recorded on the transmission system during a heat 
wave in July 1999 and all emergency procedures except for 
"manual load dump" were implemented [P/M, 2000, p. 4]. 
Voltages were low because the consumption of reactive 
power exceeded the supply [PJM, 2000]. One possible fac- 
tor contributing to this imbalance was a lack of incentive 
for generators to produce reactive power since its pro- 
duction can reduce the quantity of real power that can be 
sold (see "Interim report of the U.S. Department of 


Energy's power outage study team," 2000, http://certs.lbl. 
gov/pdf/post-interim.pdf). A root-cause analysis gave rise 
to 20 recommendations. The first recommendation was 
the development of a comprehensive voltage operating 
criteria, with one stipulation that the criteria "should 
define limits associated with non-cost actions, off-cost 
actions, and load dump" [PJM, 2000, p. 11]. 

[ 15 ] With a number of important lessons learned from 
the July 1999 event and a new energy management system 
in place, PJM considers adherence to the reactive transfer 
limits at the internal interfaces within PJM (e.g., the 
eastern interface) to be critical to the reliability of its 
system. Its training materials note that "Small increase in 
flow or load can cause large voltage fluctuation" [PJM, 
2008, p. 168] and that voltage collapse has potential to 
lead to a blackout of the system. In short, PJM reports 
that the "Reactive Transfer Limits are the most critical 
system reliability limits" because they represent the 
"largest potential system impact if exceeded" [P/M, 2008, 
p. 168]. It is therefore not surprising that an automated 
security analysis evaluates the transfer limits approxi- 
mately every 5 min and that "PJM dispatchers continu- 
ously monitor and control the flow on each transfer 
interface so that the flows remain at or below the transfer 
limits" [P/M, 2011a, p. 45]. Those readers interested in 
learning more about PJM's views on reactive power are 
encouraged to read its reactive power factsheet [P/M, 
2012a]. 

[ 16 ] PJM normally dispatches generators based on their 
cost, with the lowest cost generators being dispatched 
first. One major exception to this "economic merit," or 
"on-cost," method of dispatch is when generation is 
dispatched "out of economic merit order," or "off-cost," 
due to a transmission constraint [P/M, 2008, p. 8]. Adverse 
reactive power conditions can warrant a reduction in the 
transfer limits and thus can give rise to these events. PJM 
distinguishes this as a "reactive off-cost" operation, during 
which generators are redispatched so that the transmission 
flows at the internal interfaces remain at or below the 
reactive transfer limits. 

[ 17 ] Given that the cost of generation is a lower priority 
during a reactive off-cost operation, it is reasonable to 
suppose that these events would have a market impact. 
Consistent with this view, the average real-time price at 
PJM's Eastern Hub over the sample period was about 
$0.70 per MWh higher than at its Western Hub when the 
transmission system was in reactive on-cost status. When 
reactive off-cost operations were in effect, the average 
real-time price at the Eastern Hub was approximately 
$11.36 per MWh higher than at the Western Hub. This 
represents an approximately 20% premium above the 
average day-ahead price at PJM's Eastern Hub for those 
same market periods. This is obviously a significant 
market impact. However, the apparent market impact is 
surely modest relative to the costs imposed on consumers 
should a reactive off-cost operation not be declared and 
the system collapses as a result. 
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[is] With respect to space weather, Kappenman [2003] 
has pointed out that there is an increase in reactive power 
consumption when GICs pass through a transformer. In 
his words [Kappenman, 2003, p. 4]: 

Though these quasi-DC currents are small compared to the nor- 
mal AC current flows in the network, they have very large 
impacts upon the operation of transformers in the network.... 
The principal concern to network reliability is due to increased 
reactive power demands from transformers that can cause 
voltage regulation problems, a situation that can rapidly escalate 
into a grid-wide voltage collapse. 

[ 19 ] Consistent with this emphasis on GICs, reactive 
power consumption, and the challenge to system stability, 
Kappenman [2010, Figures 2-7] has presented evidence 
that GICs increased reactive power consumption in 
Hydro-Quebec on 13 March 1989 by a factor of approxi- 
mately 8 over the course of approximately 6 min. Power 
consumption increased from approximately 200 megavolt 
ampere reactive (MVAR) to about 1600 MVAR just 
minutes prior to the "tripping" of the five 735 kV tie 
lines needed to ensure electricity transfers to Montreal 
from the remote hydro generation facilities at James Bay 
[Kappenman, 2010]. Hydro-Quebec was unable to make 
up the loss in transmission, and the system collapsed. 
In summary, the Hydro-Quebec system collapsed on 
13 March 1989 because of GlC-induced consumption 
of reactive power. 

[ 20 ] Other researchers also have noted the impact of 
GICs on reactive power demands. For example, Molinski 
[2002] cites evidence of a linear relationship between GIC 
levels and the reactive power consumption by high-voltage 
transformers. According to Molinski, this relationship is 
evident even at low GIC levels [Molinski, 2002, Figure 9]. 

[ 21 ] PJM recognizes the challenge to reliability posed by 
space weather. In its words, "Geomagnetically-induced 
currents (GIC) caused by the solar magnetic disturbance 
(SMD) flow through the power system equipment and 
facilities may result in major increases in system reactive 
requirements, equipment damage, and disruption of 
interconnected system operation." [PJM, 2012c, p. 46]. To 
avoid these effects, PJM invokes its SMD conservative 
operations procedures. These operations are put into effect 
based on ground current readings by PJM. During these 
events, operators curtail transmission and generators are 
dispatched on the basis of system stability considerations, 
not economics. According to PJM's emergency operations 
manual, "Upon identification of a geomagnetic distur- 
bance, PJM dispatcher operates the system to geomagnetic 
disturbance transfer limits" [PJM, 2012c, p. 46]. PJM 
makes it clear that the imposition of these limits has 
implications for power generation [PJM, 2012c, p. 47]: 

When the GIC limit is approached or exceeded, generation 
redispatch assignments are made in the most effective areas to 
control this limit. PJM dispatcher also evaluates the impact of 
the existing inter-area transfers and modifies the schedules that 
adversely affect the GIC transfer limit. 


PJM details its views on solar magnetic disturbances, i.e., 
space weather, in its solar magnetic disturbances factsheet 
[PJM, 2012b]. 

[ 22 ] PJM declared its SMD procedures during the 
Bastille Day storm in July 2000. Specifically, the pro- 
cedures were implemented over the time period 15:30- 
21:07 LT on 15 July 2000. Interestingly, PJM's raw data 
files (available at http://pjm.com/markets-and-operations/ 
ops-analysis/offcostop.aspx) refer to this event as "reactive 
(SMD)" in nature. Both the transfer limits and flows were 
curtailed during this event. Problematically in terms of 
system reliability, the average transfers at PJM's eastern 
interface exceeded the limit during 1 h. 

[ 23 ] PJM replaced its emergency reporting system since 
2000 with the result being that the current system does not 
reflect the July 2000 SMD event. However, the current 
reporting system does reveal that conservative operations 
in response to solar magnetic disturbances were declared 
on four occasions over the sample period corresponding to 
this study (http://www.pjm.com/markets-and-operations/ 
etools/emerg-procedure.aspx). All of these instances 
occurred in October and November 2003. These actions 
are consistent with the NOAA SWPC assessment of the 
October-November 2003 storms (http://www.nws.noaa. 
gov/os/assessments/pdfs/SWstorms_assessment.pdf). 

[ 24 ] There was an abnormally high incidence of reactive 
off-cost operations in PJM during the Halloween storms of 
2003. Specifically, from 29 to 31 October 2003, reactive off- 
cost operations were implemented in 18 of the 72 h, a rate 
more than twice the average rate of incidence. Some 
coincided with PJM's implementation of its SMD conser- 
vative operations but others did not. During these same 
storms the Swedish high-voltage power transmission 
system also experienced problems [Pulkkinen et ah, 2005]. 
Forbes and St. Cyr [2008] also noted a series of emergency 
deployments of balancing power in the Netherlands 
power grid during these same storms and presented 
evidence that the incidence of the emergency deploy- 
ments is statistically related to the value of a GIC proxy. 
Specifically, based on 3 years of quarterly hour data, they 
report a p-value of 3.082 x 1 0 1 1 (a value corresponding to 
the odds of flipping a fair coin approximately 45 times and 
observing a "heads" each time). The null hypothesis of no 
relationship was therefore rejected. 

3. Data 

[ 25 ] A number of factors can give rise to off-cost opera- 
tions. For example, PJM has indicated that an off-cost 
operation is the most common response to violations of 
the thermal transmission limits [PJM, 2008]. This study 
focuses exclusively on reactive off-cost operations. The 
time periods in which off-cost operations were imple- 
mented are posted on the PJM website (http://pjm.com/ 
markets-and-operations/ops-analysis/offcostop.aspx). The 
raw data indicate whether the root cause of an off-cost 
operation was "reactive" in nature. As discussed in the 
previous section, PJM is able to identify the root cause 
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because it monitors reactive conditions at several inter- 
faces within its system [PJM, 2011a]. On the basis of these 
readings, all or a portion of the system is operated reactive 
on-cost or off-cost. The raw data representing reactive off- 
cost events were downloaded and a binary variable was 
created with 1 representing the outcome when all or a 
portion of the system was operated reactive off-cost for all 
or part of an hour and zero otherwise. 

[ 26 ] Following Forbes and St. Cyr [2010], we dropped 
observations from the sample if geomagnetic data were 
missing, data on forecasted load were missing for any hour 
of the day, or if ambient temperature data were missing. 
The sample also excludes transition days between stan- 
dard and daylight saving time. The hours corresponding 
to the August 2003 Northeast blackout were also excluded 
from the analysis. In total, 2,001 deletions left a sample 
of 16,262 observations to be analyzed. Over the sample 
period, one or more reactive off-cost operations were in 
effect during 1727 h, about 10% of the sample. In 476 of 
these periods, PJM also experienced constraints in its 
500 kV transformers, the incidence of which has been 
analyzed by Forbes and St. Cyr [2010]. 

[ 27 ] Data on hourly load and the locational prices by 
hour were downloaded directly from the PJM web site. 
The day-ahead hourly forecasted load data was obtained 
from the Itron Corporation (www.itron.com). The hourly 
ambient temperature data were obtained from the National 
Weather Service for the Baltimore-Washington, Pittsburgh, 
and Philadelphia airports. Data from these airports yielded 
significantly fewer missing values compared to the 
alternatives. 

[ 28 ] The econometric results reported in section 6 make 
use of day-ahead electricity price data weighted by the 
prices of coal and natural gas, the primary fossil fuels used 
by generators in PJM. These variables are included because 
they reflect expected operating conditions. Specifically, 
they proxy the opportunity costs of generators of providing 
reactive power. Economic dispatch in PJM is based on 
offers to provide generation, which, in turn, are based on 
marginal costs. In the absence of transmission constraints, 
the offers are accepted beginning with the lowest-priced 
offers. Higher-priced offers are then accepted until the 
total amount of generation offered equals the anticipated 
demand for electricity. The day-ahead price is determined 
by the expected operating characteristics of the last gen- 
erating unit that is dispatched. The day-ahead price of 
electricity relative to the price of the fuels used to gen- 
erate the electricity therefore reflects expected operating 
conditions as well as the opportunity cost of providing 
reactive power. We obtained natural gas price data from 
NGI Intelligence Press (http://intelligencepress.com); we 
obtained coal price data from Platts (http://www.platts. 
com). We transformed prices into U.S. dollars per GJ. The 
median values of the ratios are approximately 6.6 and 
25.2 for natural gas and coal, respectively. When adverse 
operating conditions are expected, the price ratio for 
natural gas can exceed 50, while the ratio for coal can 
exceed 150. 


[ 29 ] PJM has significant nuclear generation capacity and 
thus some may wonder why the price ratios discussed 
above do not include nuclear generation. The economic 
reality is that nuclear power generating units have very 
low marginal costs and a nuclear plant is highly unlikely 
to be the marginal generating unit. Consistent with this 
view, the 2006 PJM State of the market report [PJM, 2007] 
indicated that nuclear was never the marginal fuel during 
the period 2004-2006 (data before 2004 are unavailable). 
In contrast, coal and natural gas were the marginal fuels 
in more than 90% of the operating periods. 

[ 30 ] Previous literature has indicated that GIC levels in 
power grids are closely related to the time derivative of 
the horizontal component of the geomagnetic field [Bolduc 
et al, 1998; Coles et al., 1992; Makinen, 1993; Viljanen, 1998; 
Viljanen et al, 2001]. Accordingly, GICs in this study are 
proxied by the time derivative of the horizontal compo- 
nent (dH/dt) of the geomagnetic field. 

[ 31 ] For each 1-h market period, the rate of the change 
in the horizontal component of the geomagnetic field 
(dH/dt) was calculated using the largest change (in absolute 
value) in the 1-min values of horizontal component of the 
geomagnetic field over the hour in question. 

[ 32 ] The raw data were downloaded from the Inter- 
magnet website (http://www.intermagnet.org). Specifically, 
the study uses data reported by the U.S. Geological 
Survey's geomagnetic observatory in Fredericksburg (FRD), 
Virginia, which is located about 75 km south of the 
boundaries of the PJM control area during the study 
period. The peak value of dH/dt over the sample period 
was 178.4 nT/min. Based on 1-min FRD data downloaded 
from the World Data Centre for Geomagnetism, Edinburgh 
(http://www.wdc.bgs.ac.uk/catalog/master.html), this peak 
value was significantly lower than the peak of 353 nT/min 
that was obtained for FRD during the 13 March 1989 geo- 
magnetic storm that crashed the Hydro-Quebec power 
system. 

4. Preliminary Analysis Using a Nonparametric 
Test Statistic 

[ 33 ] Following Forbes and St. Cyr [2008, 2010], this study 
initially tests the hypothesis that GIC levels and the 
incidence of reactive off-cost events are related using 
Pearson's chi-square statistic, a nonparametric test statistic 
that makes no assumption about the distribution of the 
frequencies. In contrast to simple correlation analysis, it 
does not presume that the relationship is linear. 

[ 34 ] Following Sheskin [2007], we proceed by first estab- 
lishing a number of mutually exclusive categories for the 
GIC proxy. The GIC proxy data were categorized by 
quartile where GIC1, GIC2, GIC4, and GIC4 represent 
the first through the fourth quartiles, respectively. With 
respect to grid conditions, we will consider two categories: 
reactive off-cost (Off-Cost) and reactive on-cost (On-Cost). 

[ 35 ] The chi-square analysis proceeds by constructing a 
cross-tabulation between the measure of grid conditions 
and the GIC proxy and reporting on the observed 
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Table 1. Observed Frequencies for the Categories Representing 
GICs and Reactive Conditions in the PJM Power Grid, 1 April 
2002-30 April 2004 



On-Cost 

Off-Cost 

GIC1 

3819 

298 

GIC2 

3905 

417 

GIC3 

3506 

483 

GIC4 

3305 

529 


Table 3. The Ratio of Observed to Expected Frequencies for 
the Categories Representing GICs and Conditions in the 
PJM Power Grid, 1 April 2002-30 April 2004 


On-Cost Off-Cost 


GIC1 

1.038 

0.682 

GIC2 

1.011 

0.909 

GIC3 

0.983 

1.140 

GIC4 

0.964 

1.299 


frequency in each cell (Table 1). This observed frequency 
is then compared to the frequency that would be expected 
if the two variables were statistically independent 
(Table 2). The procedure for calculating these values is 
presented in Forbes and St. Cyr [2010]. Table 3 presents the 
ratio of observed to expected frequencies. Consistent with 
the hypothesis that there is a causal relationship between 
GIC levels and the incidence of reactive off-cost opera- 
tions, the reader should observe that the ratio corre- 
sponding to off-cost (on-cost) monotonically increases 
(declines) as the GIC category increases (decreases). For 
example, the observed ratio associated with GIC1 IT off- 
cost is about 32% lower than expected while the observed 
ratio associated with GIC4 T off-cost is about 30% higher 
than what would be expected under the null hypothesis 
of statistical independence. 

[36] The chi-square test can be used to assess whether 
the reported differences between the observed and expec- 
ted frequencies are statistically significant. The chi-square 
test statistic is calculated as follows: 


R C 

>= i y=i 


/ 

\ 2 



\Jl,J Jl 

•j) 


ff. 


(i) 


where /"■ represents the observed frequency, f[, represents 
the expected frequency under the assumption that the 
null hypothesis is true, R is the number of row categories, 
and C is the number of column categories. 

[ 37 ] The null hypothesis of statistical independence is 
rejected if the calculated chi-square statistic exceeds ( x, 2 )*, 
the critical value of the statistic corresponding to the level 
of statistical significance. The exact value of (\ 2 )* will 
depend on the number of degrees of freedom that will 
equal the number of row categories minus one multiplied 


Table 2. Expected Frequencies Under the Assumption of 
Statistical Independence for the Categories Representing 
GICs and Conditions in the PJM Power Grid, 1 April 
2002-30 April 2004 



On-Cost 

Off-Cost 

GIC1 

3680 

437 

GIC2 

3863 

459 

GIC3 

3565 

424 

GIC4 

3427 

407 


by the number of column categories. In this case there are 
four GIC categories and two power grid categories; thus 
there are three degrees of freedom. The corresponding 
critical value of the chi-square statistic at the 1% level of 
statistical significance equals 11.345. In this case, the cal- 
culated chi-square statistic equals 103.994, which exceeds 
the critical value of the statistic by a substantial margin. 
The associated p-value equals 2.151E-22. This value is 
approximately equal to the probability of flipping a fair 
coin 72 times and observing a "head" on each flip. Most 
individuals who flip a coin that is claimed to be "fair" 
72 times and observe a "head" each and every time would 
most likely conclude that the coin is not fair. In any event, 
the p-value in this case is well below 0.01 and thus the 
null hypothesis of statistical independence between the 
GIC proxy and the incidence of reactive off-cost events is 
rejected at the 1% level of statistical significance. This 
finding does not "prove" that GICs contribute to the 
implementation of reactive off-cost events, but it is never- 
theless consistent with a causal relationship. In the next 
section, we explore this issue more rigorously taking into 
account that there are other possible drivers of these 
events. 

5. A Multivariate Model of Reactive Off-Cost 
Events 

[ 38 ] In this section of the paper, we examine the relation- 
ship between the GIC proxy and the metric of challenged 
reliability using the multivariate logit model specification 
commonly used by researchers in making statistical infer- 
ences with respect to the incidence of binary events. This 
specification is similar to the probit methodology employed 
by Forbes and St. Cyr [2010] in that the two methodologies 
yield similar probabilities in most cases [Greene, 2008]. One 
advantage of the logit specification relative to the probit is 
that the transformed estimated coefficients are easier to 
interpret. Specifically, the predicted value of the dependent 
variable is the odds ratio, p/(l-p), where p is the probability 
of the binary dependent variable being equal to one. Each 
exponentiated coefficient represents the estimated change 
in the odds ratio corresponding to a one-unit increase in 
the corresponding predictor variable holding other vari- 
ables constant. The dependent variable in the analysis is a 
binary variable whose value equals one if a reactive off-cost 
action by the system operator occurs in hour t and equals 
zero otherwise. The explanatory variable of interest in this 
case is the GIC proxy (GIC), and thus its exponentiated 
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coefficient represents the estimated effect of a 1-unit 
increase in the GIC proxy on the odds of an off-cost event, 
all other factors being held constant. To control for possibly 
confounding factors, the model includes explanatory vari- 
ables that are presumed to directly affect or proxy the fac- 
tors that affect the probability of a reactive off-cost event. 
Exclusive of ambient temperature, scheduled electricity 
flows with other control areas and the level of actual load, 
the control variables are largely based on the findings of 
Forbes and Zampelli [2011], who present an analysis of load 
forecasting errors; load-forecasting errors being just 
another form of the energy imbalances examined in this 
study. Their analysis presented one year of out-of-sample 
evidence for the California Independent System Operator. 
The analysis indicated that the root-mean squared error 
of the day-ahead load forecast for the northern portion of 
the control area can be reduced by about 23% if the load 
forecasts are revised. Such revision would be based on 
various measures of the day-ahead load forecast and 
outcomes in the day-ahead electricity market. Each of 
these factors is discussed below. 

5.1. Ambient Temperature 

[ 39 ] We hypothesize that the probability of a reactive 
off-cost operation is higher, the higher the ambient 
temperature (T). This is consistent with the findings of 
Forbes and St Cyr's [2010] analysis of transformer con- 
straints in PJM's 500 kV transmission system. 

5.2. Actual Load, Forecasted Load, and the Intraday 
Variability in Forecasted Load 

[ 40 ] We hypothesize that the probability of an off-cost 
operation is higher the higher the level of both actual load 
(AL) and forecasted load (FL), all other factors being held 
constant. The rationale for this hypothesis is that reactive 
power is consumed by transmission lines and transfor- 
mers, which are more heavily utilized when load is higher. 
It is further hypothesized that the probability of a reactive 
off-cost operation is lower the higher the intraday vari- 
ability in forecasted load as measured by the coefficient of 
variation (CVFL). The rationale for this hypothesis is that 
a high degree of variability in the hourly forecasted load 
can be expected to favor the scheduled dispatch of more 
flexible generating units relative to base load units. This 
may have implications for the geographic distribution of 
generation relative to load, which in turn may have reactive 
power ramifications. The skewness in the intraday fore- 
casted load is also hypothesized to be a useful predictor. 
Positive skewness is represented by PSFL which equals 
the skewness in the hourly forecasted load for each day 
when the skewness is positive. It is equal to zero other- 
wise. The variable NSFL equals the absolute value of the 
skewness in the hourly forecasted load for each day 
corresponding to hour t when the skewness in the fore- 
casted load is negative. It is equal to zero otherwise. We 
hypothesize that the probability of an off-cost operation is 
higher the higher the absolute value of this measure. The 
rationale for this hypothesis is that insufficient levels of 


reactive power may be more likely on those days when 
there are several hours of unusually low levels of power 
generation. 

5.3. Scheduled Imports and Exports 

[ 41 ] The variable SI represents scheduled net imports 
during those hours in which PJM was a net importer. It is 
equal to zero otherwise. We hypothesize that higher reli- 
ance on imports increases the probability that a reactive 
off-cost operation will be implemented. The rationale for 
this hypothesis is that imports represent electricity flows 
that consume reactive power without the generation that 
supplies reactive power. The variable SE represents 
scheduled net exports during those hours in which PJM 
was a scheduled net exporter. It is equal to zero otherwise. 
We hypothesize that increases in net scheduled exports 
reduces the probability that PJM would implement a 
reactive off-cost operation. 

5.4. The Day-Ahead Electricity Price Relative 
to Fuel Costs 

[ 42 ] Coal and natural gas are the primary fossil fuels 
used to produce electricity in PJM. The model employs the 
variable DPG, defined as the ratio of the hourly day-ahead 
system price of electricity relative to the price of natural 
gas. The model also uses the variable DPC, which is the 
hourly day-ahead system price of electricity relative to 
the price of coal. Following from economic theory, we 
hypothesize that these day-ahead price ratios are useful 
proxies for expected operating conditions. Forbes and 
Zampelli [2011] have presented evidence on this point. We 
also hypothesize that the probability of an off-cost opera- 
tion is higher, the higher these day-ahead price ratios. 

5.5. Relative Fuel Prices 

[ 43 ] Generating plants in PJM that utilize coal to produce 
electricity tend to be located greater distances from 
demand centers as compared to plants that produce elec- 
tricity using natural gas. Under on-cost conditions, gen- 
erators are dispatched based on costs which are significantly 
determined by fuel prices. Thus, because reactive power 
does not travel well, changes in the price of natural gas 
relative to coal may have implications for the incidence of 
reactive off-cost events. To account for this, the model 
includes the ratio of the natural gas price relative to the coal 
price (PGC) as an explanatory variable. 

5.6. The Intraday Variability in the Day-Ahead Prices 

[ 44 ] We hypothesize that the probability of a reactive 
off-cost operation is higher, the higher the coefficient of 
variation in the day-ahead hourly prices (CVDP). The 
rationale for this hypothesis is that the coefficient of var- 
iation in the hourly prices may be a useful predictor of 
operational uncertainty. 

[ 45 ] It is also hypothesized that the probability of a 
reactive off-cost event is influenced by the degree of 
skewness in the intraday prices. When prices are nega- 
tively skewed, generators with limited flexibility in terms 
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of output (e.g., nuclear) may overproduce relative to the 
scheduled level during the periods of low prices to ensure 
the ability to meet demand in the lucrative peak load 
periods. When prices are positively skewed, the modal 
price may be insufficient to justify economically full 
adherence to the generation schedule. Because generation 
is a source of reactive power, we hypothesize that that 
odds of a shortfall in reactive power, i.e., the odds of a 
reactive off-cost event, is lower (higher), the higher the 
negative (positive) skewness in the hourly prices. The 
degree of skewness is represented by the variables PSDP 
and NSDP. PSDP equals the skewness in the hourly day- 
ahead prices for each day when the skewness in the hourly 
prices is positive. It is equal to zero otherwise. NSDP 
equals the absolute value of the skewness in the hourly 
day-ahead prices for each day when the skewness in the 
hourly prices is negative. It is equal to zero otherwise. 

5.7. Known Transmission Constraints 

[46] Following Forbes and St Cyr [2010], known trans- 
mission constraints can be measured by the absolute value 
of the difference in the day-ahead hourly prices between 
PJM's Eastern and Western hubs (DCC). We hypothesize 
that the probability of an off-cost operation is higher, the 
higher this day-ahead measure of congestion costs. 

5.8. Other Factors 

[ 47 ] Other factors can influence the probability of a 
reactive off-cost event. To capture the influence of 
unobserved hour-of-the-day effects, this study employs 
binary variables to represent each hour of the day (Hk 
where k = 2, 3, 4 ... 24) exclusive of hour 1, the effect of 
which is reflected in the overall constant term. Having 
hour 1 be reflected in the constant term avoids the 
problem of singularity in the matrix used to generate the 
estimates. Unobserved hour-of-the-day effects are con- 
trolled for by binary variables to represent each day of 
the week (Dj where j = 1, 3, 4, 5, 6, 7), exclusive of 
Monday, the effect of which is reflected in the overall 
constant term. Reflecting one of the days of the week in 
the constant term avoids the problem of singularity in 
the matrix used to generate the estimates. 

[ 48 ] In its most general form, the model is given by: 

p =f(Dj,H k , T, FL, CVFL, PSFL , NSFL , SI, SE, DPG, DPC. PGC, 
CVDP, PSDP , NSDP. DCC, GIC) , (2) 

where p is the probability of a reactive off-cost event. 

[ 49 ] The estimation of (2) was conducted using the logit 
formulation supplemented by the multivariable fractional 
polynomial (MFP) modeling approach. This technique is 
useful when one suspects that some or all of the relation- 
ships between the dependent variable and the explanatory 
variables are nonlinear [Royston and Sanerbrei, 2008] but 
there is little or no basis, theoretical or otherwise, on which 
to select a particular functional form. The MFP approach 
begins by estimating a model that is strictly linear in the 


explanatory variables. Subsequent estimations then cycle 
through a battery of nonlinear transformations of the 
explanatory variables (positive and negative powers, nat- 
ural logarithms, etc.) until it obtains the specification that 
best predicts the dependent variable. In our case, the 
analysis provided support for including 11 of the explan- 
atory variables with powers other than unity. The vari- 
ables in question are T, FL, CVFL, NSFL, AL, SI, DPC, DPG, 
PSDP, NSDP, and DCC. In the case of temperature, the 
MFP suggested specification is ln(T + 15), where 15 is the 
constant needed to avoid taking the logarithm of a nega- 
tive value. For the variable PSDP, the specification is In 
(PSDP + 0.001), where 0.001 is a constant needed to avoid 
taking the logarithm of zero. All the other MFP specifi- 
cations involve the exponents —2, 0.5, and 3. For exam- 
ple, the MFP specified formulations for the variable AL is 
cubic. 

[ 50 ] The logit/MFP estimating equation is: 

2 24 

Ln(p / (l - p)) = c + 2 J 7 jtD;t + X PjHj + a\ ln(T + 15) + a 2 FL 
k= 2 )= 2 

+ 013 CVFL 3 + a A PSFL + a 5 NSFL 3 + a 6 AL 3 
+ ajSI 0 ' 5 + ugSE + ag DPC 2 + awDPG 3 
4- cvuPGC “ + 0 J 2 CVDP 4 - 0'i3 1 n f PSDP 4- .001) 
+ a u NSDP 3 + aisDCC 0 ' 5 + 6 GIC. (3) 

The transformation on the left-hand side of (3), the logit, 
takes a number that is restricted to the [0, 1] interval and 
converts it into a value that has no upper or lower bound. 
Consequently, even when the all the explanatory variables 
have unitary exponents, the estimating equation is non- 
linear, with the marginal impact of any single independent 
variable contingent on the values of the others. In contrast, 
under the more familiar ordinary least squares specifica- 
tion the probability that the action will occur is simply a 
linear function of the independent variables with no 
interaction effects. It is also possible that the ordinary least 
squares specification could yield a predicted probability 
that is negative, a result that is obviously nonsensical 
[Greene, 2008]. 

6. Estimation and Results 

[ 51 ] There is the possibility that the effect of GICs on 
the probability of a reactive off-cost event is subject to a 
threshold. To assess this model specification issue, 
equation (3) was estimated 101 times with an assumed 
GIC threshold alternatively assumed to be 0, 1, 2, 3 ... 
100 nT/min. Based on the resulting values of the McFadden 
R-squared there is no evidence of a positive threshold. 
Specifically, the value of the McFadden R-squared was 
maximized when a threshold of zero was assumed. 

[ 52 ] Table 4 presents the estimation results. The coeffi- 
cient on the GIC variable indicates that the coefficient is 
positive as hypothesized and is also statistically significant 
at the 1% level as evidenced by its p- value of less than 


8 of 14 



S05001 


FORBES AND ST. CYR: CHALLENGES TO ELECTRIC POWER RELIABILITY 


S05001 


Table 4. Estimation Results for Equation (3) 


Variable 

Coefficient 

Estimated 

Coefficient 

Exponentiated 

Estimated 

Coefficient 

T Statistic 

P-Value 

Constant 

c 

-8.066 

3.14E-04 

-11.82 

<0.001 

Hour 2 

02 

0.107 

1.113 

0.36 

0.721 

Hour 3 

03 

-0.025 

0.975 

-0.08 

0.94 

Hour 4 

0 

0.086 

1.090 

0.26 

0.795 

Hour 5 

05 

0.017 

1.017 

0.05 

0.958 

Hour 6 

06 

-0.101 

0.904 

-0.35 

0.727 

Hour 7 

07 

-0.483 

0.617 

-1.77 

0.076 

Hour 8 

08 

-0.634 

0.530 

-2.42 

0.015 

Hour 9 

09 

-0.470 

0.625 

-1.88 

0.06 

Hour 10 

010 

-0.537 

0.584 

-2.19 

0.029 

Hour 11 

0n 

-0.509 

0.601 

-2.12 

0.034 

Hour 12 

012 

-0.519 

0.595 

-2.16 

0.031 

Hour 13 

013 

-0.614 

0.541 

-2.53 

0.012 

Hour 14 

014 

-0.726 

0.484 

-2.93 

0.003 

Hour 15 

015 

-0.750 

0.472 

-3 

0.003 

Hour 16 

016 

-0.716 

0.488 

-2.88 

0.004 

Hour 17 

017 

-0.611 

0.543 

-2.47 

0.013 

Hour 18 

018 

-0.597 

0.550 

-2.47 

0.014 

Hour 19 

019 

-0.699 

0.497 

-2.9 

0.004 

Hour 20 

020 

-0.805 

0.447 

-3.36 

0.001 

Hour 21 

021 

-0.822 

0.439 

-3.39 

0.001 

Hour 22 

022 

-0.581 

0.559 

-2.41 

0.016 

Hour 23 

023 

-0.263 

0.769 

-1.08 

0.281 

Hour 24 

024 

-0.028 

0.972 

-0.11 

0.914 

Sunday 

7i 

0.001 

1.001 

0 

0.997 

Tuesday 

72 

-0.373 

0.688 

-3.37 

0.001 

Wednesday 

73 

-0.249 

0.780 

-2.3 

0.021 

Thursday 

74 

-0.376 

0.687 

-3.33 

0.001 

Friday 

75 

-0.297 

0.743 

-2.68 

0.007 

Saturday 

76 

-0.161 

0.851 

-1.04 

0.298 

Ln(T + 15) 

Oil 

1.611 

5.009 

12.71 

<0.001 

FL 

OL2 

2.22E-05 

1.000 

1.25 

0.21 

CVFL 3 

<*3 

-307.266 

0.000 

-9 

<0.001 

PSKFL 

<*4 

-3.706 

0.025 

-2.58 

0.01 

NSKFL 3 

<*5 

1.721 

5.592 

6.57 

<0.001 

AL 3 

a 6 

2.65E-14 

1.000 

7.32 

<0.001 

SI. 5 

OL7 

0.029 

1.029 

6.8 

<0.001 

SE 

a 8 

-0.002 

0.998 

-1.07 

0.283 

DPC 2 

<*9 

-23.329 

7.4E-11 

-8.69 

<0.001 

DPG 3 

aio 

0.001 

1.001 

3.53 

<0.001 

PGC~ 2 

an 

-13.463 

1.42E-06 

-11.18 

<0.001 

CVDP 

an 

4.743 

114.831 

5.77 

<0.001 

PSDP 3 

<*13 

0.061 

1.063 

4.6 

<0.001 

NSDP 3 

ai4 

-1.133 

0.322 

-3.35 

0.001 

DCC 5 

<*15 

0.123 

1.131 

5.4 

<0.001 

GIC 

6 

0.018 

1.019 

3.4 

0.001 


Number of observations 16262 

Number of positive observations 1727 

McFadden's R 2 0.211 

Percentage of correct reactive 70 

off-cost status predictions 
Percentage of correct reactive 91 

on-cost status predictions 

Log likelihood full model —4342.9228 

Log likelihood intercept only —5504.570 


0.01. In terms of magnitude, the exponentiated estimated 
coefficient indicates that holding other factors constant, a 
1-unit increase in the GIC proxy increases the odds ratio 
of a reactive off-cost event by 1. 

[ 53 ] With respect to the other independent variables, the 
coefficients on the binary variables indicate that reactive 


off-cost events are less likely during periods of known peak 
generation. For example, the coefficients corresponding 
to the variables Friday and hour 17 are negative and 
statistically significant at the 5% level as evidenced by 
their p- values being less than 0.05. The signs of estimated 
coefficients corresponding to Ln(X + 15), FL, AL, and CVFL 3 
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Table 5. A Listing of 50 Market Periods in Which the Probability of a Reactive Off-Cost Event Was Significantly Elevated by 
Space Weather 2 


Day 

Month 

Year 

Hour 

(UT) 

Predicted 
Probability 
of Off-Cost 
Operations 

Predicted 
Incremental 
GIC-Associated 
Probability 
of Off-Cost 
Operations 

Predicted 
Probability 
of Off-Cost 
Operations in 
the Absence 
of Space 
Weather 

dH/dt 

(nT/min) 

Comment 

30 

10 

2003 

19 

0.746 

0.647 

0.100 

178.4 


29 

10 

2003 

14 

0.722 

0.600 

0.122 

159 

PfM's solar magnetic disturbance 
procedures were implemented 

31 

10 

2003 

0 

0.607 

0.459 

0.148 

118.8 


31 

10 

2003 

2 

0.493 

0.411 

0.082 

129.9 


29 

10 

2003 

18 

0.437 

0.340 

0.097 

107.5 

PJM's solar magnetic disturbance 
procedures were implemented 

29 

10 

2003 

16 

0.472 

0.338 

0.134 

95.3 

PfM's solar magnetic disturbance 
procedures were implemented 

29 

10 

2003 

17 

0.455 

0.338 

0.117 

99.9 

PJM's solar magnetic disturbance 
procedures were implemented 

30 

10 

2003 

21 

0.465 

0.295 

0.170 

78.6 


30 

10 

2003 

22 

0.522 

0.291 

0.231 

70.2 


30 

10 

2003 

20 

0.339 

0.247 

0.092 

88.4 


30 

10 

2003 

23 

0.416 

0.245 

0.171 

67.4 


31 

10 

2003 

12 

0.304 

0.232 

0.072 

93.7 


30 

10 

2003 

1 

0.308 

0.212 

0.096 

77.7 


29 

10 

2003 

15 

0.327 

0.189 

0.138 

60.3 


29 

10 

2003 

13 

0.280 

0.156 

0.123 

55.2 


30 

10 

2003 

0 

0.282 

0.153 

0.128 

53.2 


29 

10 

2003 

19 

0.238 

0.152 

0.086 

65.4 


24 

10 

2003 

22 

0.337 

0.148 

0.189 

42.4 


29 

10 

2003 

12 

0.233 

0.139 

0.093 

58.7 


29 

10 

2003 

21 

0.268 

0.137 

0.131 

48.3 


23 

5 

2002 

15 

0.196 

0.126 

0.070 

64.1 


1 

8 

2002 

23 

0.838 

0.123 

0.715 

39.4 


29 

10 

2003 

20 

0.196 

0.119 

0.078 

58 


18 

8 

2002 

18 

0.902 

0.105 

0.796 

46.4 


24 

10 

2003 

15 

0.288 

0.102 

0.187 

30.9 


29 

5 

2003 

19 

0.156 

0.099 

0.057 

60.6 


24 

10 

2003 

21 

0.263 

0.094 

0.168 

30.7 


17 

4 

2002 

16 

0.718 

0.093 

0.625 

23.1 


17 

4 

2002 

15 

0.630 

0.091 

0.539 

20.5 


31 

10 

2003 

11 

0.154 

0.087 

0.067 

50.9 


29 

10 

2003 

22 

0.288 

0.087 

0.201 

25.7 


31 

10 

2003 

13 

0.177 

0.083 

0.094 

39.8 


17 

4 

2002 

18 

0.809 

0.083 

0.727 

25.4 


15 

7 

2002 

20 

0.418 

0.082 

0.336 

19.1 


17 

7 

2002 

16 

0.338 

0.082 

0.256 

21.4 


31 

10 

2003 

1 

0.159 

0.069 

0.090 

35.4 


22 

1 

2004 

1 

0.177 

0.069 

0.109 

30.9 


30 

5 

2003 

20 

0.211 

0.068 

0.143 

25.6 


30 

10 

2003 

18 

0.180 

0.065 

0.115 

28.5 


29 

10 

2003 

11 

0.132 

0.064 

0.068 

39.8 


30 

10 

2003 

16 

0.191 

0.061 

0.129 

25.1 


29 

5 

2003 

22 

0.103 

0.061 

0.042 

52.5 


20 

11 

2003 

13 

0.108 

0.061 

0.047 

48.6 


23 

5 

2002 

16 

0.129 

0.060 

0.069 

37.8 


29 

5 

2003 

20 

0.117 

0.056 

0.061 

38.6 


29 

10 

2003 

6 

0.067 

0.056 

0.011 

99.3 


29 

8 

2003 

20 

0.695 

0.055 

0.639 

13.6 


9 

9 

2003 

20 

0.207 

0.055 

0.152 

20.4 


29 

10 

2003 

23 

0.238 

0.054 

0.184 

17.7 


22 

1 

2004 

13 

0.228 

0.054 

0.175 

18.3 



“Because of independent rounding, the predicted probability in the seventh column plus the predicted incremental probability do not add up 
to equal the predicted probability in the fifth column. 
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Table 6. A Listing of 50 Market Periods in Which the Probability of a Reactive Off-Cost Event Was Not Significantly Elevated by 
Space Weather 3 


Day 

Month 

Year 

Hour 

(UT) 

Predicted 
Probability 
of Off-Cost 
Operations 

Predicted 
Incremental 
GIC-Associated 
Probability 
of Off-Cost 
Operations 

Predicted 
Probability of 
Off-Cost Operations 
in the Absence 
of Space Weather 

dH/dt 

(nT/min) 

30 

7 

2002 

20 

0.9999 

7.57E-06 

0.9998 

2.8 

30 

7 

2002 

19 

0.9998 

7.33E-06 

0.9998 

2.3 

30 

7 

2002 

18 

0.9997 

1.01E-05 

0.9997 

2 

14 

8 

2002 

19 

0.9996 

9.60E-06 

0.9996 

1.4 

30 

7 

2002 

21 

0.9996 

2.03E-05 

0.9996 

2.6 

5 

8 

2002 

20 

0.9996 

1.13E-05 

0.9996 

1.4 

14 

8 

2002 

20 

0.9981 

5.03E-05 

0.9980 

1.4 

5 

8 

2002 

19 

0.9977 

3.87E-05 

0.9976 

0.9 

30 

7 

2002 

17 

0.9976 

2.17E-04 

0.9974 

4.7 

5 

8 

2002 

21 

0.9972 

1.14E-04 

0.9971 

2.2 

31 

7 

2002 

20 

0.9970 

5.03E-05 

0.9969 

0.9 

31 

7 

2002 

19 

0.9957 

6.39E-05 

0.9956 

0.8 

15 

8 

2002 

19 

0.9955 

6.03E-04 

0.9949 

6.8 

15 

8 

2002 

20 

0.9950 

9.24E-04 

0.9940 

9.2 

14 

8 

2002 

18 

0.9946 

1.30E-04 

0.9945 

1.3 

14 

8 

2002 

21 

0.9946 

4.02E-04 

0.9942 

3.9 

15 

8 

2002 

18 

0.9920 

3.41E-03 

0.9886 

19.4 

5 

8 

2002 

18 

0.9880 

3.98E-04 

0.9876 

1.8 

15 

8 

2002 

17 

0.9852 

5.45E-04 

0.9847 

2 

16 

8 

2002 

20 

0.9847 

4.80E-04 

0.9842 

1.7 

16 

8 

2002 

19 

0.9842 

3.18E-04 

0.9839 

1.1 

15 

8 

2002 

21 

0.9840 

1.83E-03 

0.9822 

6 

1 

8 

2002 

20 

0.9827 

1.74E-03 

0.9809 

5.3 

13 

8 

2002 

20 

0.9826 

3.81E-04 

0.9822 

1.2 

14 

8 

2002 

17 

0.9822 

5.54E-04 

0.9817 

1.7 

2 

8 

2002 

19 

0.9804 

2.95E-03 

0.9775 

7.8 

1 

8 

2002 

19 

0.9779 

1.31E-03 

0.9766 

3.2 

2 

8 

2002 

20 

0.9774 

2.39E-03 

0.9750 

5.6 

30 

7 

2002 

22 

0.9770 

3.33E-04 

0.9767 

0.8 

5 

8 

2002 

17 

0.9759 

5.25E-04 

0.9754 

1.2 

16 

8 

2002 

18 

0.9733 

5.30E-04 

0.9728 

1.1 

31 

7 

2002 

21 

0.9715 

4.11E-04 

0.9711 

0.8 

1 

8 

2002 

21 

0.9710 

5.60E-03 

0.9654 

9.9 

29 

7 

2002 

21 

0.9706 

8.52E-04 

0.9697 

1.6 

13 

8 

2002 

21 

0.9701 

6.48E-04 

0.9694 

1.2 

16 

8 

2002 

21 

0.9699 

7.61E-04 

0.9692 

1.4 

29 

7 

2002 

20 

0.9699 

8.71E-04 

0.9690 

1.6 

30 

7 

2002 

16 

0.9697 

1.10E-03 

0.9686 

2 

16 

8 

2002 

17 

0.9684 

5.68E-04 

0.9679 

1 

13 

8 

2002 

19 

0.9680 

8.08E-04 

0.9672 

1.4 

2 

8 

2002 

18 

0.9660 

2.31E-03 

0.9637 

3.7 

31 

7 

2002 

18 

0.9600 

7.12E-04 

0.9593 

1 

19 

8 

2002 

20 

0.9591 

4.79E-03 

0.9543 

6.3 

1 

8 

2002 

18 

0.9573 

8.89E-03 

0.9485 

10.8 

26 

6 

2002 

20 

0.9572 

5.31E-04 

0.9567 

0.7 

14 

8 

2002 

22 

0.9557 

1.74E-03 

0.9540 

2.2 

26 

6 

2002 

19 

0.9547 

6.41E-04 

0.9541 

0.8 

3 

7 

2002 

19 

0.9510 

1.65E-03 

0.9494 

1.9 

15 

8 

2002 

22 

0.9461 

2.59E-03 

0.9435 

2.7 

2 

8 

2002 

21 

0.9429 

9.93E-03 

0.9330 

9.3 


“Because of independent rounding, the predicted probability in the seventh column plus the predicted incremental probability do not add up 
to equal the predicted probability in the fifth column. 


are consistent with expectations. Except for FL, these coef- 
ficients are also statistically significant as evidenced by their 
p-values. As hypothesized, the coefficients corresponding to 
PSFL 3 and NSFL 3 are negative and positive, respectively. 
Both coefficients are statistically significant. Also as 


hypothesized, the coefficients on the variables SI and SE are 
positive and negative, respectively. However, of the two 
coefficients, only the coefficient on SI is statistically signifi- 
cant. The coefficients corresponding to DPC~ 2 and DPG 3 are 
statistically significant and the estimated marginal impacts 
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Table 7. System Conditions, the GIC Proxy, and the Predicted Probability of a Reactive Off-Cost Event 


Vulnerability 
Percentile in the 
Absence of 
Space Weather 

Probability of an 
Off-Cost Event in 
the Absence of 
Space Weather 

Value of the GIC Proxy 
(in nT/min) That Would 
Increase the Probability 
of Reactive Off-Cost 
Event to 0.51 

Value of the GIC Proxy 
(in nT/min) That Would 
Increase the Probability 
of a Reactive 
Off-Cost Event to 0.75 

Value of the 
GIC Proxy (in nT/min) 
That Would Increase 
the Probability 
of a Reactive 
Off-Cost Event to 0.95 

5 th 

0.0055 

284 

342 

445 

10 th 

0.0108 

248 

305 

408 

25 th 

0.0279 

195 

253 

355 

50 th 

0.0643 

148 

205 

308 

75 th 

0.1215 

110 

167 

270 

90 th 

0.2110 

74 

131 

234 

95 th 

0.3318 

40 

98 

200 


are consistent with expectations when the values of the 
exponents are considered. The estimated marginal impact of 
PGC~ 2 is also consistent with expectations, and the coeffi- 
cient is highly statistically as evidenced by its p-value. 

[ 54 ] With respect to the model's explanatory power, note 
that it is not possible to calculate a conventional R-squared 
that is meaningful because the dependent variable is 
binary while the predicted values are probabilities. A 
number of alternative scalar fit measures have been 
introduced. According to Greene [2002, E15-28], these 
measures "... share the flaw that none satisfactorily mimic 
the true measure of the proportion of variation explained 
given by R 2 in the linear regression context." 

[ 55 ] McFadden's R-squared is one of the more com- 
monly reported measures of scalar fit when the dependent 
variable is binary. Its value here is 0.211. This value indi- 
cates that the log likelihood function, the objective func- 
tion whose maximization yields the estimated parameters, 
improves by about 21% compared to when the model is 
estimated with only a constant term as an explanatory 
variable. The percentage of correct predictions is a more 
meaningful measure of the scalar fit. Using 0.5 as the 
threshold for a prediction of a reactive off-cost event, the 
percentage of correct predictions when on-cost status is 
predicted equals 91%, while the percentage of correct 
predictions when off-cost status is predicted equals 70%. 
The latter percentage is approximately 90% when the 
predicted probability is greater than 0.80. 

[ 56 ] The econometric results presented in Table 4 should 
not be taken as definitive proof that GICs contribute to 
challenged reliability. Statistical methods alone cannot 
render results that establish or reject causality between 
two variables that are contemporaneously correlated. For 
this reason, ad hoc approaches to address the issue of 
causality are methodologically unsound. Instead, the results 
of inferential statistical analyses indicate whether the data 
are consistent with the hypothesis under consideration. 

[ 57 ] There are three logical explanations for the results 
presented in Table 4. First, consistent with Molinski 
[2002], Kappenman [2003], and PJM's implementation of 
its solar magnetic disturbance procedures, GICs contribute 
to adverse reactive power conditions as hypothesized. 
Second, there is reverse causation, i.e., adverse reactive 


power conditions cause the GICs. Third, GICs do not con- 
tribute to adverse reactive power conditions; instead an 
omitted variable that is highly correlated with GICs is the 
true cause. The second possibility does not appear likely 
given that solar activity and the resulting properly mea- 
sured GIC proxy are truly exogenous variables. With 
respect to the third possibility, we have yet to learn of an 
excluded terrestrial variable that contributes to adverse 
reactive power conditions and is also highly correlated 
with GICs. We suspect that others would be equally 
challenged in this regard. In short, the results pre- 
sented here would not be easily dismissed by anyone with 
knowledge of the effects of GICs on reactive power con- 
sumption and formal training in multivariate inferential 
statistics. Others are free to adopt their own criteria. For 
example, some may consider our results acceptable only 
if the relationship is visually apparent using a scatter 
diagram or graph. Yet many important relationships do 
not meet this "visually apparent" criterion, e.g., smoking 
and lung cancer, a relationship that was established in 
large part using inferential statistics [Parascandola et al., 
2006]. While a scatter diagram can illuminate a relation- 
ship between the variables X and Y when the Y is a function 
of only X, it can fail miserably when Y is a function of more 
than one variable. 

[ 58 ] Table 5 presents a listing of 50 market periods in 
which the predicted probability of a reactive off-cost event 
was significantly elevated by space weather. This listing 
reports the predicted probability based on terrestrial con- 
siderations along with the predicted probability based on 
terrestrial plus space weather considerations. Depending 
on geomagnetic conditions, the difference in the two pre- 
dicted probabilities can be significant. For example, based 
on terrestrial conditions only, the predicted probability of 
a reactive off-cost event in hour 14 of 29 October 2003 
equals 0.122; when the impact of geomagnetic conditions 
is factored in, the predicted probability is 0.722. It may be 
worth noting that PJM's solar magnetic disturbance pro- 
cedures were implemented during this market period. 
For these 50 h, the median predicted probability of an 
off-cost event (0.2849) is more than double the predicted 
probability when only terrestrial determinants of the events 
are considered (0.1259). To put the results in perspective. 
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Table 6 presents a listing of 50 market periods in which the 
predicted probability of a reactive off-cost event was not 
significantly elevated by space weather. 

[ 59 ] Table 7 presents the results of an analysis of the 
predicted probabilities. The predicted probabilities in the 
absence of the GIC proxy were first calculated and sorted. 
At the fifth percentile of the predicted probabilities, the 
terrestrial-based probability of a reactive off-cost event is 
0.0055, while at the 95th percentile the probability equals 
0.3318. Table 7 then reports the value of the GIC proxy that 
would be needed to increase the predicted probability to 
0.51, 0.75, and 0.95. At the 5th percentile of the terrestrial- 
based probabilities (column two), the GIC proxy would 
need to equal 284, 342, and 445 nT/min. for the predicted 
probability of a reactive off-cost event to rise to 0.51, 0.75, 
and 0.95, respectively. In contrast, at the 95th percentile of 
the terrestrial-based probabilities the GIC proxy would 
only need to equal 40, 98, and 200 nT/min. for the pre- 
dicted probability of an off-cost event to rise to 0.51, 0.75, 
and 0.95, respectively. 

7. Conclusion 

[ 60 ] This paper has addressed whether geomagnetic 
activity was a challenge to power system reliability during 
solar cycle 23 for the PJM regional transmission organiza- 
tion in North America. We examined the relationship 
between a GIC proxy and a metric of challenged reliability 
using the logit statistical method with statistical controls for 
expected system conditions. The results indicate that while 
there is no evidence of a space weather induced blackout 
in PJM during solar cycle 23, PJM's operations were nev- 
ertheless challenged by geomagnetic activity. The findings 
are consistent with previous literature. The results are also 
consistent with PJM's implementation of its solar mag- 
netic disturbance procedures during the Bastille Day storm 
in July 2000 as well as during the Halloween Storms in 
October 2003. 

[ 61 ] The analysis contained in this paper also indicates 
that the GlC/reactive power vulnerability of the power 
system is contingent on system conditions at the time of 
the geomagnetic storm. A storm would truly need to be a 
super storm to induce a reactive off-cost event when the 
terrestrial-based vulnerability of the system is low. In 
contrast, our results indicated that a modest geomagnetic 
storm could easily give rise to a reliability challenge when 
the terrestrial-based vulnerability of the system is high. 
This finding may have implications for mitigating the 
overall electric power reliability challenge. While space 
weather forecasting is in its infancy, the results of this 
analysis suggest that it is possible to forecast the terrestrial- 
based vulnerability of the power system. Inspection of 
Table 6 suggests that such forecasts may have the potential 
to enhance reliability even when the role of space weather 
is minor. These forecasts may also have the potential to 
better inform system operators about the space weather 
vulnerability of their systems, which would position them 
to make better use of space weather forecasts. 
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